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Abstract 

We consider nonlinear event systems with quantized state information and de- 
sign a globally stabilizing controller from which only the minimal required number 
of control value changes along the feedback trajectory to a given initial condition 
is transmitted to the plant. In addition, we present a non-optimal heuristic ap- 
proach which might reduce the number of control value changes and requires a 
lower computational effort. The constructions are illustrated by two numerical 
examples. 

1 Introduction 

Traditionally, controllers for (nonlinear) systems have been designed using a continuum 
as the underlying time domain. With the rise of digital information processing, time- 
triggered or sampled-data controller designs have become popular. There, a regular 
grid of time instances serves as the time domain, cf. [AW97]. Both schemes close the 
control loop independently of the system's behavior. This might lead to unnecessary 
communication between the controller and the plant. In the case that the communication 
is implemented via a digital network, restrictions like the maximal bandwith or load 
dependent stochastic effects might play a role and influence the behavior of the closed 
loop system. In order to decrease the network load and possibly avoid these effects, in 
event based control, information is only transmitted when necessary in order to ensure 
stability of the closed loop system, see, e.g., [Arz99, OMT02, AB02, TW06, KB06, 
VK06, HSVDB07, Ast08, GJ08, LL09]. 
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Another means of reducing the amount of data which has to be transmitted (and thus 
further reducing the network load) is to use a quantization of an underlying continuous 
state space. While any real number which is transmitted digitally, necessarily comes 
from a quantized set since only finitely many digits can be transmitted, here one aims 
for quantizations which are as coarse as possible since then fewer bits will suffice to 
encode the data. We refer to, e.g., [Hsu92, Lun94, FJL02, Sch03]. 

Recently, a new approach for the construction of controllers for quantized systems has 
been proposed which relies on a set oriented approach in combination with graph theo- 
retic techniques, cf. [JO04, GJ07, GJ08]. In [GM09], this approach has been extended 
to event systems. 

In the present contribution we describe how to extend this approach such that the 
number of times that data has to be transmitted from the controller to the plant is 
minimized along the feedback trajectory to some initial condition. The construction is 
based on the optimality principle with a suitably chosen state space and cost function. 
Additionally, we present a non-optimal heuristic approach which also reduces the number 
of data transmission events while requiring a significantly lower computational effort. 
The two constructions are illustrated by two numerical examples, an inverted pendulum 
and a thermofluid batch process. 

The paper is structured as follows: In Section 2 we briefly summarize the basic con- 
struction from [GJ08, GM09]. In Section 3 we describe how to extend this construction 
such that the number of data transmissions from the controller to the plant is minimized 
and illustrate this construction by two numerical examples in Section 4. Finally, in Sec- 
tion 5, we propose the heuristic scheme for reducing the communication effort. Again, 
this is illustrated by the two examples from Section 4. 

2 Global optimal feedbacks for quantized nonlinear 
event systems 

In this section, we summarize the constructions from [GJ08, GM09]: We are given a 
plant which is modeled by a nonlinear discrete time control system (which may, e.g., be 
derived from a continuous time system by time sampling) 

x(k + l) = f(x(k),u(k)), fc = 0,1,2,..., (1) 

where / : X x U — > X is continuous, x(k) G X is the state and u[k) G U is the control 
input, X C W 1 and U C M m compact. In addition to /, we are given a continuous 
running cost function c : X x hi — > [0, oo) as well as a a target set X* C X. We 
assume c to satisfy c(x,u) = iff x G X*. Our goal is to compute a feedback law 
for this system which drives the system into the target set X* while minimizing the 
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accumulated cost. However, the information which is transmitted from the plant to the 
controller is restricted in the following two ways: 

1. Event model: The controller only receives information on the state whenever an 
event occurs. Formally, based on the discrete time model (1) of the plant, we are 
dealing with the discrete time system 

x(£+l) = f(x(£),u(£)), £ = 0,1,..., (2) 

where 

f(x,u) = f^ u \x,u), (3) 

r : X x U — > No is a given event function and the iterate f r is defined by 
f°(x,u) = x and f r (x,u) = f(f r ~ 1 (x,u),u), cf. [GM09]. Accordingly, we define 
an associated running cost c : X x U — > [0, oo) by 

r(x,u) — 1 

c{x,u) = c{f k {x,u),u). (4) 

k=0 

Note that we can reconstruct the "true time" k from the "event time" £ by the 
event function r: we have that 

k(£+l) = k{£) + r{x{£),u{£)). 

2. Quantization model: The controller only receives quantized information on the 
state. Formally, we are given a (finite) partition P = \V\, . . . ,Vd}, Vi C X, of 
X which induces an equivalence relation ^ on X x X by x ^ y ^ x and y lie in 
the same partition element. We denote by [x] G P the corresponding equivalence 
class of x G X. Only [x(£]] is transmitted from the plant to the controller at the 
event time £. Thus, from the viewpoint of the controller, the plant is given by the 
finite state system, cf. [GJ08, GM09] 

V(£ + l) = F(V(£),u(£), 1 (£)), £ = 0,1,..., (5) 

defined by 

F(P,u,j) = [Kl{V),u% VeP,ueU, 

where 7 : P — > X denotes a choice function which satisfies [7("P)] = V for all 
V G P. The choice function models the fact that it is unknown to the controller 
from which exact state x(£) the system transits to the next cell V{1 + 1). We let 
r denote the set of those functions. Thus, in each step, the dynamics is influenced 
by the two control parameters u(£) and ^{t). Our goal is to choose u(£) in each 
step such that the state V{£) is controlled into the target set. At the same time, 
the influence of the choice function may be viewed as a perturbation which might 
prevent us from reaching X*. In this sense, (5) constitutes a dynamic game, cf. 
[GJ08]. 
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2.1 Computing the optimal feedback 

In order to be compatible with our quantization, from now on we assume that X* is 
given by the union of some elements from P. For the quantized system (5) we define 

C(V,u) := sup c(x,u). 

For given P(0) eP,w= (u(£))t E U N and 7 = ("y(£))e E T N , the cost accumulated along 
the associated trajectory (V(£))e E P N of (5) (which depends on u and 7) is 

L 

J(P(0),u,t) : =£c(p(4«(*)) e [0,oo], 

e=o 

where L = min{£ > : V(£) C X*} E NU {00}. The (upper) optimal value function is 

V(V) :=sup inf J(P,u,T(u)) E [0,oo], 
r mgw n 

where T : U N — )■ T N is a strategy of the form 

r(ti) = r((ii(€))^) 

= ( 7l (u(l)),7 2 (u(l),u(2)),73(u(l),u(2),u(3)),...) 

and the sup in the definition of the optimal value function is over all strategies of this 
form. The optimal value function - by standard arguments, cf. [Ber95] - for V in the 
stabilizable set S = {V E P \ V(P) < 00} is the unique solution to the optimality 
principle 



V(P) = inf <^ C(V, u) + sup V(F(V, u, 7)) } (6) 

together with the boundary condition V(P) = for V C X*. Given V, we obtain an 
optimal feedback for (5) by setting 



u{V) := argmin \ C{V, u) + sup V(F(P, u, 7)) \ (7) 

for P E S. Note that we also immediately obtain a feedback for the original system 
(1) resp. (2) from this by setting u(x) := w([£c]) for x E {J VeS F. By construction, any 
trajectory of the closed loop system 

x(k + l) = f(x(k),u([x(k)])) (8) 

with x(0) E S is eventually reaching the target set X*, cf. [GJ07, GM09]. 

Computationally, the feedback is constructed using a directed, weighted hypergraph 
together with a corresponding shortest path algorithm, cf. [JO04, GJ08, vL07] (see also 
Section 5). 
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3 Construction of a lazy feedback 



When the data transmission between the plant and the controller is realized via a digital 
network it is often desirable to minimize the amount of transmitted information in order 
to reduce the overall network load. More specifically, here we treat the question of how 
to minimize the number of times that a new control value has to be transmitted from 
the controller to the plant. Using an optimization based feedback construction, this goal 
can directly be modelled by suitably defining the running cost function. 

In order to detect a change in the control value generated by the controller we need to 
be able to compare to its value from the previous time step (resp. event). We therefore 
define the extended state space Z := X xU. Based on the event system (2), we consider 
the event system 

z(£+l)=g(z(£),u(i)), £ = 0,1,2,... (9) 

with z{i) = (x(£),w(£)) G Z the extended state vector, u(£) G U and g : Z xU — > Z 
is defined by 

f(x,u) 



9(z,u) = g((x,w),u) 



u 



We define Z* := X* x U as the target set in the extended state space so that reaching 
the target only depends on the state x. We further define an associated running cost 
function d : Z X U — > [0, oo) by 

d{{x, to), u) = (1 - X)c(x, u) + A(l - 5(u - to)) (10) 

with 

'<»>-{£: ir ' (») 

Here, A G [0, 1) must be strictly < 1 in order to guarantee that d(z, u) = iff z G Z*. 
Note that the dynamics of the extended system (9) does not depend on the second 
component w of the extended state vector z, but only the modified cost function d 
does. 

We can now apply the construction from the previous section to the system (9) with 
cost function (10). To this end, we would need to construct a partition of U. Instead, in 
order to simplify the exposition, here we simply assume that U is discrete, i.e. contains 
only finitely many elements. We then use PxWas the underlying partition for the 
quantization. We denote the resulting optimal value function by V\ : Z — > [0, oo], the 
stabilizable subset by 

S\ '■= {z G Z : V x ([z\) < oo} 
and the associated feedback by u\ : S\ — > U. 
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We will show that for a sufficiently large A < 1 the closed loop system 

z(£ + l) = g(z(£),u x (^))), e = 0,1,2,..., (12) 

is asymptotically stable on S\, i.e. that for 2(0) G S x the associated trajectory enters 
Z* in finitely many steps. Furthermore, the number of control value changes along this 
trajectory will be minimal. 

To be more precise: For some initial state z = z(0) G S\ let (z(£))i G Z N , z{£) = 
(x(£),w (£))), be the corresponding trajectory of the closed loop system (12), let 

L(z,u x ) = min{£ > : z{£) G Z*} 

be the number of time steps until the trajectory reaches the target set Z*, 

L(z,u x ) 

E(z,u x ) = J2 l-S(u x (z(£))-w(£)) 
the number of control value changes along the corresponding trajectory as well as 

L(z,u x ) 

J(z,u x )= c(x(£),u x (z(£))) 

1=0 

the accumulated (original) costs. 

Theorem 1. For all \e[0,l), S xU C S x and x{£) X* as i -> 00. 

Further, there exists a A < 1 such that for any feedback u : S x — >• W for the extended 
system and z G S x with L(z,u) < 00 holds E(z,u) > E(z,u x ). 

Proof. By definition, the extended system (9) and the cost function (10) fulfill the as- 
sumptions in [GJ07], so asymptotic stability of the closed loop system (12) directly 
follows for all z G S x by their proof. 

In order to show that S x hi C 5a for all A G [0, 1), choose A G [0, 1) and some initial 
value 2(0) = (x(0),u(0)) G S xW arbitrarily. Consider the feedback 

u(z) = u((x,u)) := u(x) 

for system (9), where u(x) denotes the feedback for (2) which has been constructed in 
Section 2. This leads to a trajectory (x(£), u(£))i of the extended system with (x(£))i 
being exactly the trajectory of the original system (2). Since x{fS) G S, V^a^O)) is finite 
and the accumulated cost J(z(0), u) for this trajectory does not exceed (1 — \)V(x(0)) + 
AL(z(0), w) which is finite. According to the optimality of V x , 

V x (z(0)) < (1 - X)V(x(0)) + XL(z(0),u) < 00 



6 



follows, i.e. z(0) G S x . 

To show the optimality of u x with respect to the number of control value changes, 
assume there exists a feedback u : S x — >• U with E(z, u) < E(z, u x ) — 1 for some z G S x . 
Since u x is optimal, the following inequality holds: 



(1- X)J{z,u)+\E{z,u) (13) 

> (l-\)J(z,u x ) + \E(z,u x ) (14) 

> (l-X)J(z,u x ) + XE(z,u) + X (15) 

and thus 

(1 - X)J(z,u) > (1 - X)J{z,u x ) + A. 

As the costs denoted by J are finite, A — > 1 leads to a contradiction. □ 

4 Numerical experiments 
4.1 Nonlinear Inverted Pendulum 

For our numerical experiments we first consider an inverted pendulum on a cart, cf. 
[JYH01]. The motion of the pendulum is given by the continuous time control system 

4 2 \ m r 2 g m r 

m r cos 09 </3 H (i> sin 2o? — - sin o? = — n — - cos cp, 

3 / 2 I ml 

where (99, 0) G [0, 27r] x R denotes the state of the pendulum and u ElA C E is the control 
input. We have used the parameters m = 2 for the pendulum mass, m r = m/ (m + M) 
for the mass ratio with cart mass M = 8, I = 0.5 as the length of the pendulum and 
g = 9.8 for the gravitational constant. As the instantaneous cost function, we employ 

, 0.01 2 / x 

q{<P,<P,u,t) = —^-u +t, (16) 

where t G 1R is the system's time. Denoting the system's evolution operator for constant 
control functions u(t) = u G U by $'(a?,w), a? = (ip,ip), we consider the discrete time 
system f(x,u) = <& T (x,u) for T = 0.01, i.e., the sampled continuous time system 
with sampling rate T = 0.01. The map <fr T is approximated via the classical Runge- 
Kutta scheme of order 4 with 5 equidistant steps. The discrete time cost function is 
obtained by numerically integrating the continuous time instantaneous cost according 
to c(x,u) = J Q T q(^ t (x,u),u,t)dt. We choose the state space X = [—10,10] x [—8,8] 
and a partition of 2 7 x 2 7 equally sized partition elements on X. The control space is 
chosen as U = [—64,64], discretized by 17 equidistant samples and the target region is 
set to [— |, |] x |] (i.e. 8x8 partition elements around [0,0]). 
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Let s(x) G M 2 and t(x) G K 2 denote the center and the radius of the rectangular 
partition element containing x, respectively, then we define an event set via 



(3(x) = {y = (y 1 ,y 2 ) T <E X : \y { - Si(x)\ < e r ti(x), i = 1, 2} 
with event radius e r = 9 and the event function 



(17) 



min{t G 0.01N : **(aj,u) £ 
r(x\ u) = < if not empty (18) 
0, else. 

For this quantized event system, we employ the construction in the two previous sections, 
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Figure 1: Inverted pendulum: Control sequence over time (left) and the associated 
trajectories in state space for the initial state (ir + 0.5, 0) T . Blue (dark): generated by 
the ordinary feedback, red (light): generated by the lazy feedback. 



i.e. we compute the ("ordinary") feedback as described in Section 2 as well as the lazy 
feedback from Section 3 with A = 0.99. 

Figure 1 shows two trajectories of the closed loop system (8) starting at the initial 
state (it + 0.5, 0) T G X . On the right-hand side we show these two trajectories in state 
space while on the left-hand side we plot the associated control sequences over time. The 
blue (dark) trajectory results from the ordinary feedback, while the red (light) trajectory 
is generated with the lazy feedback. 
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The gray ticks on the left indicate all time instances where an event occurs. While 
the ordinary feedback leads to a change of the control value at almost every event and 
generates a total number of 32 control value changes, the lazy feedback stabilizes the 
system using only 5 control changes. Note that the time until the target set is reached 
(which is essentially minimized here) remains almost the same. 

4.2 Batch Reactor 

In this numercial experiment, the aim is to control a thermofluid process in a batch 
reactor (cf. Figure 2) as described in [GJJ + 10]. The main part of the process consists in 
the cylindrical batch reactor TB which has a continuously adjustable inflow via valve Vi 
of water from the spherical tank T3 above. In addition, a permanent outflow only 
depending on the fluid level in TB is present. Heating rods can increase the temperature 




Figure 2: Thermofluid process 

of the fluid in TB while cooling can only be achieved by the inflow of cool water from T3. 
The two state variables that can be continously measured are the fluid level /tb and the 
temperature -#tb in TB, so the state becomes x = (xi,x^) = (/tb,"#tb)- The valve 
angle U\ G [0,1] of valve Vi and the power U2 G {0, ...,6} of the heating rods are 
considered as input u = (ui, u?). To model the nonlinear dynamics of the process we 
use the following differential equation with parameters from Table 1 (cf. [GJJ + 10]): 
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X 

*2 = T/ / \ ( <?T3(Wl)Or3 - ^2) + , 

where 

7-l(r 6 (ll.lw 2 + 13.1m + 0.2) for u > 0.2, 
else (20) 

V(x) = 0.07a; - 1.9- 10 3 for x > 0.26. 

The unit of the flow is m 3 /s and the unit of the volume V of the fluid in TB is m 3 . 

Table 1: Parameters and constants 



<?T3(WJ 



Parameter Value Meaning 



Pel 


3000 W 


Electrical power 




0.84 J/(Ws) 


Heat transfer coefficient 


c p 


4180 J/(kgK) 
9.81 m/s 2 


Heat capacity of water 


9 


Gravitation constant 


Q 


998 kg/m 3 


Density of water 


$T3 


293.15 K 


Temperature of inflow 


K A 


1.59- 10" 5 m 3 /m 


Outflow parameter 


A h 


0.07m 2 


Cross sectional area 



The goal is to steer the system into a neighbourhood of the operating point (Jtb, ^tb) = 
(0.349 m, 310.56 K) with only a minimum number of control changes, therefore we use 
a weighted sum of time and a quadratic function in x and u with a small gain as cost 
function. According to the physical limitations of the reactor, we set the state space 
to X = [0.26,0.45]m x [293.15, 323.15]K and use a discretization of 2 6 x 2 6 partition 
elements. The continuous input u\ G [0, 1] is discretized via 12 equidistant samples. For 
the time integration of the ordinary differential equation, we use the classical Runge- 
Kutta scheme of order 4 with 5 equidistant steps and a time step of Is. An event is 
generated whenever the state leaves a partition element, i.e. we employ an event radius 
of e r — 1 here. 

We consider the initial state [0.275m, 295K] and - like in the first example - compare 
two trajectories associated, respectively, to the ordinary feedback described in Section 2 
and the lazy one as described in Section 3 with A = 0.9. In Figure 3 (left) we compare 
the generated control sequences. Using the lazy feedback, the number of control value 
changes is reduced dramatically from 13 to only 2. Note that, again, the time required 
for the system to reach the target set remains almost the same, cf. the right part of 
Figure 3. 
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20 40 60 80 100 120 140 0.27 0.28 0.29 0.3 0.31 0.32 0.33 0.34 0.35 0.36 

k X - 1 

Figure 3: Left: Output of the controller over time for the batch process with initial 
state [0.275m, 295K]. Upper figure: using the standard cost function. Lower figure: lazy 
feedback, i.e. minimal number of control value changes (A = 0.9). Right: Associated 
feedback trajectories in state space. Blue (dark): using the standard cost function. Red 
(light): lazy feedback, i.e. minimal number of control value changes (A = 0.9). 

5 A heuristic approach for reducing the data trans- 
mission frequency 

One drawback of the lazy feedback construction proposed in Section 3 is the need to 
extend the state space to X x U. This leads to a notable increase in both memory 
requirements and computing time. In this section we will illustrate that this expansion 
is really needed in order to obtain the minimum number of control changes. Neverthe- 
less, there is a heuristic way of reducing the number of control changes without these 
drawbacks. In order to derive this heuristic, we first have a closer look at the graph 
construction which is used in order to compute the value function and the feedback, cf. 
[JO04, GJ08]. 

For each partition element V G P, the value V(V) is given by the length of the 
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shortest path from P to the set of partition elements which constitute X* in the following 
hypergraph G = (P, £): The edges of G are given by 

s = {(v, f(v, u, r)) | p g p u g u} 

weighted by 

W (V, A) = inf {C(P, u):ueU, F(V, u, T) = A}, 

where F(V,u, V) := {F(V,u, 7) G P | 7 G T}. As such, it can be computed by an 
efficient Dijkstra-type algorithm [GJ07, vL07], cf. Algorithm 1. 

Algorithm 1 MinMax-Dijkstra 
Input: hypergraph (P, £) 

weights w : £ — > (0, 00) 

set of target nodes O C P 
Output: value function V : P ^ [0, 00] 

control input w : P — > £Y 

1: for all P G P\0 do 

2: 1/(P) := OO 

3: for all P G O do 

4: 1/(P) := 

5: Q := P 

6: while Q 7^ do 

7: P : = argmiUp/gQ V{V) 

8: Q ;= Q\{p} 

9: for all (Q, N) G £ with P G N do 

10: if A c P\Q then 

11: if V(Q) >w(Q,N) + V(V) then 

12: F(Q) :=W (Q,A) + V(P) 

13: u(Q) := u(Q, N) 



Here, u(Q,A) = argmin u6W {C(P, u) \ F(Q,u,T) = A}. One easily shows that 
when a node P is removed from Q in line 6, V(V) and «(P) are fixed until termination 
(cf. for example [AMOT90]). It follows that when a hyperedge (<2, A) is being processed 
in lines 9 and 10, the value V(V) is fixed for all target nodes P G A of the hyperedge 
because A C P\Q (line 7, 8 ). This information can be used for the choice of a proper 
control u(Q). 

To be more precise, let u(Q, A) denote the control applied for hyperedge (Q,A), 
Uq(V) an arbitrary control for the target nodes P G O and A G [0, 1) a parameter. Then 
in order to reduce the number of control changes we change lines 3 — 4 and lines 11 — 13 
in Algorithm 1 as follows: 
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3: for all V e O do 

4: V{V) : = and u{V) : = u (V) 



11: if V(Q) > (1 - X)w{Q, N) + Xa(u(Q, N), N) + V(V) then 
12: V(Q) := (1 - X)w{Q, N) + Xa(u(Q, N),N) + V(V) 
13: u(Q):=u(Q,N) 



Here, a : U x ^ [0, oo) is a function dependent on the control of the hyperedge 
and the controls of the possible subsequent states. In order to reduce the number of 
control changes along trajectories, a(u, N) has to be large when u ^ u(V) for many 
V G iV and should be small otherwise. For example, one can set 

a(u, N) = 1 - 5 ^argmax V{V) \ - u(Q, N)\ 

with 5 defined in (11). In our numerical tests, however, choosing 

seemed to lead to better results. 
5.1 A counterexample 

In order to show that the heuristic approach in general does not lead to a feedback 
which produces the minimal number of control value changes, we consider the following 
counterexample, cf. Figure 4. 




Figure 4: Counterexample to illustrate that the heuristic approach in general does not 
lead to a feedback which minimizes the number of control changes 

We start with node on the right and assume that control input u\ is optimal 
for node V4. Let V3 be the neighbour of V4 being processed next in Algorithm 1. As 
control input U\ does not lead to a transition to V4 we only have the choice between w 2 
and 113. Since both of them lead to a control change, we may chose as well 1*3. This 
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procedure can be done recursively up to state V\ on the left with u\ and u 2 or u± and 
U3, respectively. We end up with a trajectory from V\ to P4 switching control 3 times, 
which is not the optimum compared to having constant control U2 and switching control 
only once at the end. 

The reason for this effect is that for each node V we only have computed the optimal 
value V(V) and control input u(V) with respect to a trajectory that starts at V. But 
we do not know about optimality when V is not the initial state of a trajectory. A list 
of the values V(V) for all pairs (V, u) G P xW would be sufficient to remedy this - but 
this then leads directly back to the lazy feedback approach of section 3. 

5.2 Numerical experiments 

Nevertheless, in numerical experiments we often obtain reasonable results even with the 
heuristic approach. The number of control changes often may be reduced without en- 
larging the state space and so without any noticeable effect on the memory consumption 
(cf. Figure 5, where the number of control value changes was reduced from 33 to 18). 

On the other hand, the heuristic approach does not seem to be prone to failure, as 
shown in repeating the numerical experiment with the batch reactor. Here (cf. Figure 6), 
the number of control value changes is actually increased. A closer inspection seems to 
reveal a phenomenon similar to the counterexample described above. 

6 Conclusion 

Based on an extended model of a nonlinear quantized event system we introduced the 
concept of a lazy feedback which stabilizes the system with the minimal number of 
control value changes. We show that the lazy feedback is indeed optimal in this sense 
among all feedbacks which stabilize the same set of initial conditons. In addition, we 
illustrated a heuristic method which sometimes can be used for reducing the number of 
control changes without any further computational effort compared to the lazy feedback 
construction. 
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